73 research outputs found
Fast Low-Rank Matrix Learning with Nonconvex Regularization
Low-rank modeling has a lot of important applications in machine learning,
computer vision and social network analysis. While the matrix rank is often
approximated by the convex nuclear norm, the use of nonconvex low-rank
regularizers has demonstrated better recovery performance. However, the
resultant optimization problem is much more challenging. A very recent
state-of-the-art is based on the proximal gradient algorithm. However, it
requires an expensive full SVD in each proximal step. In this paper, we show
that for many commonly-used nonconvex low-rank regularizers, a cutoff can be
derived to automatically threshold the singular values obtained from the
proximal operator. This allows the use of power method to approximate the SVD
efficiently. Besides, the proximal operator can be reduced to that of a much
smaller matrix projected onto this leading subspace. Convergence, with a rate
of O(1/T) where T is the number of iterations, can be guaranteed. Extensive
experiments are performed on matrix completion and robust principal component
analysis. The proposed method achieves significant speedup over the
state-of-the-art. Moreover, the matrix solution obtained is more accurate and
has a lower rank than that of the traditional nuclear norm regularizer.Comment: Long version of conference paper appeared ICDM 201
Decoupling Representation and Classifier for Noisy Label Learning
Since convolutional neural networks (ConvNets) can easily memorize noisy
labels, which are ubiquitous in visual classification tasks, it has been a
great challenge to train ConvNets against them robustly. Various solutions,
e.g., sample selection, label correction, and robustifying loss functions, have
been proposed for this challenge, and most of them stick to the end-to-end
training of the representation (feature extractor) and classifier. In this
paper, by a deep rethinking and careful re-examining on learning behaviors of
the representation and classifier, we discover that the representation is much
more fragile in the presence of noisy labels than the classifier. Thus, we are
motivated to design a new method, i.e., REED, to leverage above discoveries to
learn from noisy labels robustly. The proposed method contains three stages,
i.e., obtaining the representation by self-supervised learning without any
labels, transferring the noisy label learning problem into a semisupervised one
by the classifier directly and reliably trained with noisy labels, and joint
semi-supervised retraining of both the representation and classifier. Extensive
experiments are performed on both synthetic and real benchmark datasets.
Results demonstrate that the proposed method can beat the state-of-the-art ones
by a large margin, especially under high noise level
- …